Efficient, balanced data placement algorithm in scalable storage clusters

نویسنده

  • LIU Zhong
چکیده

Data distribution and load balancing become increasingly important in large-scale distributed storage system. This paper focuses on the problem of designing an optimal, self-adaptive strategies for balanced distribution and reorganization of replicated objects among a dynamically heterogeneous nodes, and presents a novel decentralized algorithm, Dynamic Interval Mapping, which maps replicated objects to a scalable collection of nodes, it distributes objects to nodes optimally, redistributing minimum amount of objects when new nodes are added or existing nodes are removed to maintain the balanced distribution. It supports weighted allocation and guarantees that replicas of a particular object are not placed on the same node. The time complexity and storage requirements are superior to previous methods.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

RUSH: Balanced, Decentralized Distribution for Replicated Data in Scalable Storage Clusters

Typical algorithms for decentralized data distribution work best in a system that is fully built before it first used; adding or removing components results in either extensive reorganization of data or load imbalance in the system. We have developed a decentralized algorithm, RUSH (Replication Under Scalable Hashing), that maps replicated objects to a scalable collection of storage servers or ...

متن کامل

RDIM: A Self-adaptive and Balanced Distribution for Replicated Data in Scalable Storage Clusters

As storage systems scale from a few storage nodes to hundreds or thousands, data distribution and load balancing become increasingly important. We present a novel decentralized algorithm, RDIM (Replication Under Dynamic Interval Mapping), which maps replicated objects to a scalable collection of storage nodes. RDIM distributes objects to nodes evenly, redistributing as few objects as possible w...

متن کامل

A Bit-Window based Algorithm for Balanced and Efficient Object Placement and Lookup in Large-scale Object Based Storage Cluster

Business requirements for data availability, survivability, and performance have driven the need for building the network storage that interconnects various kinds of storage devices to allow remote access by multiple hosts. A new revolutionary storage technology called “Object based Storage Devices (OSD)” is now emerging as a promising technology to meet the high performance needs and to addres...

متن کامل

Scalable, Balanced Model-based Clustering

This paper presents a general framework for adapting any generative (model-based) clustering algorithm to provide balanced solutions, i.e., clusters of comparable sizes. Partitional, model-based clustering algorithms are viewed as an iterative two-step optimization process—iterative model re-estimation and sample re-assignment. Instead of a maximum-likelihood (ML) assignment, a balanceconstrain...

متن کامل

An Efficient Secret Sharing-based Storage System for Cloud-based Internet of Things

Internet of things (IoTs) is the newfound information architecture based on the internet that develops interactions between objects and services in a secure and reliable environment. As the availability of many smart devices rises, secure and scalable mass storage systems for aggregate data is required in IoTs applications. In this paper, we propose a new method for storing aggregate data in Io...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2007